Answerer in Questioner's Mind for Goal-Oriented Visual Dialogue

We propose an “answerer in questioner’s mind” (AQM) framework, a novel approach for goal-oriented dialogue. In AQM, a questioner asks and infers based on an approximated probabilistic model of the answerer in the questioner’s model. The questioner figures out the answerer’s intent by selecting a plausible question following the principle of information theory, which explicitly calculates the information gain of candidate intentions and candidate answers to each question. This process allows the proposed method to overcome learning difficulties, owing to comparative neural conversation systems and a deep reinforcement learning approach. We test our framework on two goal-oriented visual dialog tasks, “MNIST Counting Dialog” and “GuessWhat?!.”